{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "06_PyTorch",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RvwFrkzSPbw7",
        "colab_type": "text"
      },
      "source": [
        "# PyTorch\n",
        "\n",
        "In this notebook, we'll learn the basics of [PyTorch](https://pytorch.org), which is a machine learning library used to build dynamic neural networks. We'll learn about the basics, like creating and using Tensors."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0aqN-ffaP4t1",
        "colab_type": "text"
      },
      "source": [
        "<div align=\"left\">\n",
        "<a href=\"https://github.com/madewithml/basics/blob/master/notebooks/06_PyTorch.ipynb\" role=\"button\"><img class=\"notebook-badge-image\" src=\"https://img.shields.io/static/v1?label=&amp;message=View%20On%20GitHub&amp;color=586069&amp;logo=github&amp;labelColor=2f363d\"></a>&nbsp;\n",
        "<a href=\"https://colab.research.google.com/github/madewithml/basics/blob/master/notebooks/06_PyTorch.ipynb\"><img class=\"notebook-badge-image\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
        "</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SFa_PSr2tvaC",
        "colab_type": "text"
      },
      "source": [
        "# Set seeds"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eLAkqoRKtyFD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l9krh147uJOV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "SEED = 1234"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1uLEnBgft22Y",
        "colab_type": "code",
        "outputId": "81fb9f46-9b61-4858-a591-aee25b32f74b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Set seed for reproducibility\n",
        "np.random.seed(seed=SEED)\n",
        "torch.manual_seed(SEED)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7fd54bfb9190>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "08AUKP9xu8YQ",
        "colab_type": "text"
      },
      "source": [
        "# Basics"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o3jBRfYZuNqF",
        "colab_type": "code",
        "outputId": "f40c3f56-ef66-45c4-e1cc-140f5267a44c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Creating a random tensor\n",
        "x = torch.randn(2, 3) # normal distribution (rand(2,3) -> uniform distribution)\n",
        "print(f\"Type: {x.type()}\")\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Type: torch.FloatTensor\n",
            "Size: torch.Size([2, 3])\n",
            "Values: \n",
            "tensor([[ 0.0461,  0.4024, -1.0115],\n",
            "        [ 0.2167, -0.6123,  0.5036]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pho5A7JluNvj",
        "colab_type": "code",
        "outputId": "2904d1ef-5e23-4fd2-d8ac-e90d9c2b415b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Zero and Ones tensor\n",
        "x = torch.zeros(2, 3)\n",
        "print (x)\n",
        "x = torch.ones(2, 3)\n",
        "print (x)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[0., 0., 0.],\n",
            "        [0., 0., 0.]])\n",
            "tensor([[1., 1., 1.],\n",
            "        [1., 1., 1.]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UTecl1RduNtL",
        "colab_type": "code",
        "outputId": "4980a540-a66a-4e98-a9e8-0849827efb58",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# List → Tensor\n",
        "x = torch.Tensor([[1, 2, 3],[4, 5, 6]])\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 3])\n",
            "Values: \n",
            "tensor([[1., 2., 3.],\n",
            "        [4., 5., 6.]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2OQTnxWOuNnY",
        "colab_type": "code",
        "outputId": "9b172f99-a46b-4311-9367-0f6894ff2666",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# NumPy array → Tensor\n",
        "x = torch.Tensor(np.random.rand(2, 3))\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 3])\n",
            "Values: \n",
            "tensor([[0.1915, 0.6221, 0.4377],\n",
            "        [0.7854, 0.7800, 0.2726]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8K2kWrkZuilf",
        "colab_type": "code",
        "outputId": "8049a7a6-c565-4c16-a6a0-27fa89a58751",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "# Changing tensor type\n",
        "x = torch.Tensor(3, 4)\n",
        "print(f\"Type: {x.type()}\")\n",
        "x = x.long()\n",
        "print(f\"Type: {x.type()}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Type: torch.FloatTensor\n",
            "Type: torch.LongTensor\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6LxCmxqFu6sq",
        "colab_type": "text"
      },
      "source": [
        "# Operations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yfYLm_1Buixy",
        "colab_type": "code",
        "outputId": "12a99436-64ac-42e1-89be-409f23adf757",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Addition\n",
        "x = torch.randn(2, 3)\n",
        "y = torch.randn(2, 3)\n",
        "z = x + y\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 3])\n",
            "Values: \n",
            "tensor([[ 0.1824, -1.3555, -1.0664],\n",
            "        [ 1.0333,  0.1368, -0.1310]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "22abfI18uiuw",
        "colab_type": "code",
        "outputId": "b82398cd-cc4d-4d8a-9255-03d0fa49ef7a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "# Dot product\n",
        "x = torch.randn(2, 3)\n",
        "y = torch.randn(3, 2)\n",
        "z = torch.mm(x, y)\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 2])\n",
            "Values: \n",
            "tensor([[ 0.1410,  0.2433],\n",
            "        [-2.0222, -0.0703]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p1ztJNrruiqv",
        "colab_type": "code",
        "outputId": "b7c10009-0881-4c14-c643-ba5003853948",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        }
      },
      "source": [
        "# Transpose\n",
        "x = torch.randn(2, 3)\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = torch.t(x)\n",
        "print(f\"Size: {y.shape}\")\n",
        "print(f\"Values: \\n{y}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 3])\n",
            "Values: \n",
            "tensor([[ 0.5797, -0.0599,  0.1816],\n",
            "        [-0.6797, -0.2567, -1.8189]])\n",
            "Size: torch.Size([3, 2])\n",
            "Values: \n",
            "tensor([[ 0.5797, -0.6797],\n",
            "        [-0.0599, -0.2567],\n",
            "        [ 0.1816, -1.8189]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zoLDryFYuioF",
        "colab_type": "code",
        "outputId": "8c3668f3-b521-48fc-8e22-51977cddc17a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "# Reshape\n",
        "x = torch.randn(2, 3)\n",
        "z = x.view(3, 2)\n",
        "print(f\"Size: {z.shape}\")\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([3, 2])\n",
            "Values: \n",
            "tensor([[0.2111, 0.3372],\n",
            "        [0.6638, 1.0397],\n",
            "        [1.8434, 0.6588]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2fdNmFu3vlE7",
        "colab_type": "code",
        "outputId": "2083c766-8960-43e4-a4f9-d389e7046db2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 612
        }
      },
      "source": [
        "# Dangers of reshaping (unintended consequences)\n",
        "x = torch.tensor([\n",
        "    [[1,1,1,1], [2,2,2,2], [3,3,3,3]],\n",
        "    [[10,10,10,10], [20,20,20,20], [30,30,30,30]]\n",
        "])\n",
        "print(f\"Size: {x.shape}\")\n",
        "print(f\"x: \\n{x}\\n\")\n",
        "\n",
        "a = x.view(x.size(1), -1)\n",
        "print(f\"\\nSize: {a.shape}\")\n",
        "print(f\"a: \\n{a}\\n\")\n",
        "\n",
        "b = x.transpose(0,1).contiguous()\n",
        "print(f\"\\nSize: {b.shape}\")\n",
        "print(f\"b: \\n{b}\\n\")\n",
        "\n",
        "c = b.view(b.size(0), -1)\n",
        "print(f\"\\nSize: {c.shape}\")\n",
        "print(f\"c: \\n{c}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([2, 3, 4])\n",
            "x: \n",
            "tensor([[[ 1,  1,  1,  1],\n",
            "         [ 2,  2,  2,  2],\n",
            "         [ 3,  3,  3,  3]],\n",
            "\n",
            "        [[10, 10, 10, 10],\n",
            "         [20, 20, 20, 20],\n",
            "         [30, 30, 30, 30]]])\n",
            "\n",
            "\n",
            "Size: torch.Size([3, 8])\n",
            "a: \n",
            "tensor([[ 1,  1,  1,  1,  2,  2,  2,  2],\n",
            "        [ 3,  3,  3,  3, 10, 10, 10, 10],\n",
            "        [20, 20, 20, 20, 30, 30, 30, 30]])\n",
            "\n",
            "\n",
            "Size: torch.Size([3, 2, 4])\n",
            "b: \n",
            "tensor([[[ 1,  1,  1,  1],\n",
            "         [10, 10, 10, 10]],\n",
            "\n",
            "        [[ 2,  2,  2,  2],\n",
            "         [20, 20, 20, 20]],\n",
            "\n",
            "        [[ 3,  3,  3,  3],\n",
            "         [30, 30, 30, 30]]])\n",
            "\n",
            "\n",
            "Size: torch.Size([3, 8])\n",
            "c: \n",
            "tensor([[ 1,  1,  1,  1, 10, 10, 10, 10],\n",
            "        [ 2,  2,  2,  2, 20, 20, 20, 20],\n",
            "        [ 3,  3,  3,  3, 30, 30, 30, 30]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HcW6i9xJwU2Q",
        "colab_type": "code",
        "outputId": "0cb8f6f0-4f62-4424-8554-b47fcf09e49e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "# Dimensional operations\n",
        "x = torch.randn(2, 3)\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = torch.sum(x, dim=0) # add each row's value for every column\n",
        "print(f\"Values: \\n{y}\")\n",
        "z = torch.sum(x, dim=1) # add each columns's value for every row\n",
        "print(f\"Values: \\n{z}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "tensor([[-0.2349, -0.0306,  1.7462],\n",
            "        [-0.0722, -1.6794, -1.7010]])\n",
            "Values: \n",
            "tensor([-0.3071, -1.7100,  0.0452])\n",
            "Values: \n",
            "tensor([ 1.4806, -3.4525])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kqxljkudzH0M",
        "colab_type": "text"
      },
      "source": [
        "# Indexing, Splicing and Joining"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q8-w1Cb3wsj0",
        "colab_type": "code",
        "outputId": "ae1ad2f2-2c89-496f-9cba-221d7bec8995",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "x = torch.randn(3, 4)\n",
        "print (f\"x: \\n{x}\")\n",
        "print (f\"x[:1]: \\n{x[:1]}\")\n",
        "print (f\"x[:1, 1:3]: \\n{x[:1, 1:3]}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x: \n",
            "tensor([[ 0.6486,  1.7653,  1.0812,  1.2436],\n",
            "        [ 0.8971, -0.0784,  0.5548, -0.0845],\n",
            "        [ 0.5903, -1.0032, -1.7873,  0.0538]])\n",
            "x[:1]: \n",
            "tensor([[0.6486, 1.7653, 1.0812, 1.2436]])\n",
            "x[:1, 1:3]: \n",
            "tensor([[1.7653, 1.0812]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jBGk_740wsm3",
        "colab_type": "code",
        "outputId": "9dfafe5a-4e2b-47d1-b1f3-c49278a21195",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Select with dimensional indicies\n",
        "x = torch.randn(2, 3)\n",
        "print(f\"Values: \\n{x}\")\n",
        "\n",
        "col_indices = torch.LongTensor([0, 2])\n",
        "chosen = torch.index_select(x, dim=1, index=col_indices) # values from column 0 & 2\n",
        "print(f\"Values: \\n{chosen}\") \n",
        "\n",
        "row_indices = torch.LongTensor([0, 1])\n",
        "col_indices = torch.LongTensor([0, 2])\n",
        "chosen = x[row_indices, col_indices] # values from (0, 0) & (2, 1)\n",
        "print(f\"Values: \\n{chosen}\") "
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "tensor([[-0.4572,  0.0901,  0.4018],\n",
            "        [-1.1542,  0.1192, -0.7348]])\n",
            "Values: \n",
            "tensor([[-0.4572,  0.4018],\n",
            "        [-1.1542, -0.7348]])\n",
            "Values: \n",
            "tensor([-0.4572, -0.7348])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UI_hboLNwsqQ",
        "colab_type": "code",
        "outputId": "24125e4f-ac96-49df-e0ea-69d35027de38",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Concatenation\n",
        "x = torch.randn(2, 3)\n",
        "print(f\"Values: \\n{x}\")\n",
        "y = torch.cat([x, x], dim=0) # stack by rows (dim=1 to stack by columns)\n",
        "print(f\"Values: \\n{y}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Values: \n",
            "tensor([[-1.5864, -0.2671,  2.6874],\n",
            "        [-0.4633,  0.6639,  0.2383]])\n",
            "Values: \n",
            "tensor([[-1.5864, -0.2671,  2.6874],\n",
            "        [-0.4633,  0.6639,  0.2383],\n",
            "        [-1.5864, -0.2671,  2.6874],\n",
            "        [-0.4633,  0.6639,  0.2383]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lK1OQUYL1bE3",
        "colab_type": "text"
      },
      "source": [
        "# Gradients"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Ft6PAeW0WCe",
        "colab_type": "code",
        "outputId": "6d56d9cc-a1df-4afd-9aa7-446e835f7bbd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 153
        }
      },
      "source": [
        "# Tensors with gradient bookkeeping\n",
        "x = torch.rand(3, 4, requires_grad=True)\n",
        "y = 3*x + 2\n",
        "z = y.mean()\n",
        "z.backward() # z has to be scalar\n",
        "print(f\"x: \\n{x}\")\n",
        "print(f\"x.grad: \\n{x.grad}\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "x: \n",
            "tensor([[0.0417, 0.0214, 0.6696, 0.0336],\n",
            "        [0.0903, 0.7542, 0.7427, 0.0444],\n",
            "        [0.8598, 0.4028, 0.8712, 0.2066]], requires_grad=True)\n",
            "x.grad: \n",
            "tensor([[0.2500, 0.2500, 0.2500, 0.2500],\n",
            "        [0.2500, 0.2500, 0.2500, 0.2500],\n",
            "        [0.2500, 0.2500, 0.2500, 0.2500]])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VF5Q5kfs1rXZ",
        "colab_type": "text"
      },
      "source": [
        "* $ y = 3x + 2 $\n",
        "* $ z = \\sum{y}/N $\n",
        "* $ \\frac{\\partial(z)}{\\partial(x)} = \\frac{\\partial(z)}{\\partial(y)} \\frac{\\partial(y)}{\\partial(x)} = \\frac{1}{N} * 3 = \\frac{1}{12} * 3 = 0.25 $"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kseQSKj72H8S",
        "colab_type": "text"
      },
      "source": [
        "# CUDA tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZE-ZyECv0WOX",
        "colab_type": "code",
        "outputId": "d497625f-a7d0-4150-a6f6-8bd6d64e3b37",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Is CUDA available?\n",
        "print (torch.cuda.is_available())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "False\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n5sWo3Yv2MxO",
        "colab_type": "text"
      },
      "source": [
        "If False (CUDA is not available), let's change that by following these steps: Go to *Runtime* > *Change runtime type* > Change *Hardware accelertor* to *GPU* > Click *Save*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ewamITzX2W-B",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IwsrvGad2NDO",
        "colab_type": "code",
        "outputId": "9770c0f0-40bc-45ce-8725-d9842eab1158",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Is CUDA available now?\n",
        "print (torch.cuda.is_available())"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "50ewrqUVCRHg",
        "colab_type": "code",
        "outputId": "e72f5517-dc4e-4304-9618-eb04168c95a2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "# Set device\n",
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
        "print (device)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "cuda\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s12ivWJZCLq7",
        "colab_type": "code",
        "outputId": "ac5dde7a-fd72-4c80-c6c1-ad2cd19b912b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "x = torch.rand(2,3)\n",
        "print (x.is_cuda)\n",
        "x = torch.rand(2,3).to(device) # sTensor is stored on the GPU\n",
        "print (x.is_cuda)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "False\n",
            "True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xr1Vsnq7CLpB",
        "colab_type": "text"
      },
      "source": [
        "---\n",
        "Share and discover ML projects at <a href=\"https://madewithml.com/\">Made With ML</a>.\n",
        "\n",
        "<div align=\"left\">\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://github.com/madewithml/basics\"><img src=\"https://img.shields.io/github/stars/madewithml/basics.svg?style=social&label=Star\"></a>&nbsp;\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://www.linkedin.com/company/madewithml\"><img src=\"https://img.shields.io/badge/style--5eba00.svg?label=LinkedIn&logo=linkedin&style=social\"></a>&nbsp;\n",
        "<a class=\"ai-header-badge\" target=\"_blank\" href=\"https://twitter.com/madewithml\"><img src=\"https://img.shields.io/twitter/follow/madewithml.svg?label=Follow&style=social\"></a>\n",
        "</div>\n",
        "             "
      ]
    }
  ]
}